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The Framework for Educational Research 


At the present time, whether we like it or not, most of the research 
being done in education consists of individual studies, largely autonomous, 
difficult to synthesize. Any one study may be a sparkling gem, but there 
is a marked absence of comprehensive designs for organization of the 
many contributions. The research is largely supported by the resources 
of motivated individuals, who do their studies at personal sacrifice. As a 
result, the total field of problems tends to be covered in a manner that is 
apparently democratic, free, and inefficient. 


It is interesting to consider what might be done. It has been suggested F 
at various times that specific types of changes should be made. Some § 
suggest that the individual institution should specialize, so as to produce § 


research on particular problems, in an organized pattern. Others feel that 


the key to progress lies in the promotion of large projects, financed ade- F 
quately by foundations, directed by a leader who coordinates the work 
of a staff of specialists. Others, who seem to feel that research is always § 
ultimately done by those persons who have a special type of personality, § 
favor continuation of a free program, suggesting that penetration can best § 
be achieved through persistence by individuals in lines of research dictated F 


by their personal interests. 


Perhaps some of the more general problems will be clarified if more § 


persons support research by their own participation and by other types 


of contributions. In spite of current criticism, research in education is 9 
likely to continue for a long time to be characterized by a heavy emphasis F 


upon fact-finding, the initial exploration of many little problems in a 
descriptive manner, replication of studies by accident rather than by 
design, and, finally, the production of research reports by a very small 
minority of workers in education. Is this what is wanted by the persons 
most concerned? H.D.C. 
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An Evaluation of 
Doctoral Dissertations in 


Educational Administration in California 


Howarbp S. BrETSCH 


The study included an analysis and an evaluation of 104 doctoral 
dissertations in Educational Administration which had been accepted by 
four major universities in California between 1940 and 1952. 

The development of criteria for selecting and evaluating the disserta- 
tions constituted the first major step in the study. The second major step 
consisted of reviewing each dissertation and applying the criteria to it. 


Selection of the Dissertations 


The four areas from which dissertations were selected were: Educational 
Finance, School District Organization, School Housing, and Staff Personnel. 
The dissertations had been accepted by Stanford University, University of 
California at Berkeley and Los Angeles, and the University of Southern 
California. Six criteria were developed for selecting dissertations, the 
most significant of which were concerned with the relationships of the 
studies to the content and functions of Educational Administration. Certain 
dissertations in other disciplines bore important relationship to the field 
of Educational Administration but were omitted from this study. 


Criteria for Evaluating Dissertations 


Unlike most of the criteria found in the literature those employed in 
this study emphasized the relative importance of the various parts of the 
dissertation and were not developed as a check list which assumed coordi- 
nate value for each item. The full statement of the criteria constituted 
several pages. Essentially, however, each dissertation was evaluated on 
the basis of (1) the significance of the dissertation, (2) the appropriateness 
and excellence of the procedures, and (3) the strength of the presentation. 





Howard S. Bretsch was a member of the faculty of the University of California 
at Berkeley until last summer. He then went to the University of Michigan as 
Professor of Educational Administration. He has also been on the faculty of the 
University of New Hampshire and the summer faculty of Pennsylvania State 
University. His public school experience involved work as a teacher and adminis- 
trator in New York. The work on which this article is based was done at Berkeley 
in 1952-1955. The work was supported by research funds from the University of 
California. 
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Of these three, the significance of the dissertation was regarded as primary, 


agers nificance emphasized creativity, originality, the estab. 
peng —of—salationships._ and the importance of thé~fnal contribution, 7 


Oughout the evaluation, stress was placed on the quality of thinking | 


that had gone into producing the research and the explicitness of the 
author. The criteria of significance were stated as questions. However, in 
order to be classified as significant, the dissertation had to fulfill one or 
more of the criteria in the affirmative. Stated briefly the criteria of signifi 
cance were: 


1. Did the study establish new truths or add new knowledge to the f 


field? 


2. Did the researcher develop a new technique? 


( identifying and measuring significant phenomena? 


4. Did the researcher, through a synthesis of factors heretofore iso- f 


lated introduce a new field of study? 


5. Did the researcher develop valid criteria for judging theory or § 


practice? 


6. Did the researcher provide insight into solving practical problems | 


effectively? 


The emphasis on newness or originality was regarded as defensible | 


since the creativity and sound imagination that produce an original work 
very frequently demonstrate, in themselves, a high level of achievement. 
This emphasis on originality did not exclude the possibility that a student 
might have produced a very significant piece of work by digging more 


deeply into an old problem or attempting to verify earlier studies on a ff 


given subject. 


The procedures of the dissertation were evaluated by giving special f 
attention to (1) the design for the study, (2) the selection of the subject, § 
(3) the validity of the assumptions, (4) the use of related materials, (5) the F 


techniques for the collection and treatment of the data, (6) the insightful- 


ness of the writer in interpreting the data, and (7) the extent to which the F 


judgments and the conclusions were supported by the evidence. The pres- 
entation was evaluated largely on the organization, the data, and the 
format of the dissertation. 


The findings of the study were of two types: (1) An evaluation of each k 


dissertation based on an application of the criteria, (2) A group analysis 


of the dissertations for each of the four areas. Neither the evaluations of 
each dissertation nor the group analysis can be given in this brief report. 7 
However, the observations and implications which follow indicate the 
dominant importance of evaluation to the study and provide insight into 7 


the nature of it. 
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Findings of This Study 


1. The significance of most of the dissertations lay in helping to solve 
practical problems. Many of them were limited in general significance 
since they dealt primarily with practical problems of a given locale. 


2. In general, the studies did not deal with the broader social implica- 
tions for education, nor did they, except rarely, add important new knowl- 
edge to the field. 


3. There appeared to be a trend in some institutions for doctoral candi- 
dates to engage in studies in which they were admittedly pioneering new 
fields or making new approaches to established ones. Many of the studies 
of this type demonstrated a high quality of thinking both in the develop- 
ment of the problem and in the pursuit of it. 


4, Limitations upon the researchers seemed to have raised an unsolved 
dilemma. In some cases the researcher chose a highly significant problem, 
the importance of which he was not fully aware. As a result, he did not 
employ the necessary skills for pursuing it nor engage in a very penetrating 
interpretation of the findings. In other cases the researcher chose a less 
important problem and because of its simplicity completed it very satisfac- 
torily. The decision to select a problem of outstanding importance rather 
than one of less significance is not always easy to make. However, the 
extent to which the capability of the candidate can be matched with the 
difficulty of the problem seems important to the quality of the research 
produced. 


5. Most of the studies did not test hypotheses. 


6. With few exceptions, the studies were of the descriptive type. Of 
these, it would seem that too many of the normative studies did not attempt 
to test the reliability of the data. 


7. Experimental variation as a method of research was not employed 
to any great degree by researchers in educational administration. Admit- 
tedly the problem of studying human behavior experimentally is difficult. 
However, in the opinion of the writer, this difficulty does not exonerate 
researchers from two things: First, attempting to use experimental variation, 
wherever appropriate, as it has been known traditionally as a research 
method; and second, finding-newer and multiple ways for conducting 
experimental studies. 


8. Nearly all of the researchers employed a variety of techniques for 
the collection and the treatment of data. In general this was sound, espe- 
cially so if they were used to establish the validity and reliability of data. 
However, some researchers seemed to have thought that the use of a 
variety of techniques for collecting a large bulk of data compensated for 
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not having selected an important study. In this manner, the multiple 
approach merely provided more work for the researcher. 

9. For the most part, researchers were careful to identify their assump. 9 
tions. They were not so careful, however, in their use of them. 

10. In those studies which sought to establish the reliability and 
significance of statistical relationships, sampling procedures were reasonably 
well carried out. This was not true, however, in the normative type studies, 

11. Related materials were used effectively as background material, 
but rarely were they used to verify the procedures of the study or justify 
the attempt to extend previous investigations. 

12. Criteria were developed and employed in about a fifth of the 
dissertations. The strength of the criteria lay in testing the validity of them 
through logical thinking or submitting them to a jury of experts. In some 
cases, however, the researcher did not seem aware of the limitations of the J 
“accredited jury” technique nor of the nature of criteria. j 

13. None of the studies was successful in determining cause and effect 7 
relationships. Rather, most of the studies of relationship were those in 
which associational relationships were established. 

14. In general, agency sponsorship tended to result in the constructing 
of a better instrument and in securing a broader sampling of data. Only 
rarely was it apparent that agency sponsorship restricted the freedom of 
the researcher. 

15. Few studies were specific extensions of earlier studies, nor did 
many of them attempt to test or validate previous research. Most of them, 
however, did point out problems which needed further study. 

16. The researcher who was most rigorous in the application of sound 
research methods and therefore had reason to be most confident of the 
results of his study was cautious in the pronouncement of conclusions. 
However, the researcher who did not apply rigorous tests to his data and 
was less careful, skillful, and probably less capable in the use of sound 
research methods frequently cited a long list of conclusions. 























Implications for Researchers 








The following implications are focused on improving research in educa- 
tional administration. Additional observations will be cited only as they 
help to provide the setting for the implication. 

1. Rarely is it possible for an individual researcher in a single study 
to resolve all of the issues of an important problem. Rather, significant 
results usually emerge from painstaking research carried on over long 
periods of time, during which many approaches are made to the problem. 
These conditions for significant research have bearing upon the selection 
of problems by doctoral candidates. Had more of the reseachers chosen — 
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to extend or verify the work of the earlier researchers the attempt to close 
a serious gap in the research in educational administration would have 
been undertaken and it is probable that the final results of the studies would 
have been more significant. 


2. Essentially each doctoral candidate must choose between selecting 
an important problem or selecting one of less significance. It would seem 
that the doctoral candidate should attempt to make an accurate appraisal 
both of his own potential for research and the significance of the problem. 
In the opinion of the writer, the researcher should attempt to overcome 
as many of the practical obstacles as possible in the preliminary exploration 
of the problem. Then, if the pursuit of a significant problem seems feasible, 
the researcher should hold himself accountable for making as great a 
contribution as he is capable of making. The researcher who does not 
pursue a significant problem nor one that challenges his highest capabilities 
seems to this writer to have abdicated his responsibilities as a doctoral 
candidate. On the other hand, it may be only slightly less desirable for 
the candidate to attempt a problem which he knows lies beyond his capacity. 


3. Many of the dissertations reported in this study would have been 
more significant had it been possible to draw from them broader social 
implications. It appeared that some doctoral candidates launched into their 
studies with insufficient background for them. This resulted in some cases 
in the selection of unimportant problems or in not discovering the deeper 
meanings of a significant problem. The implication for the researcher is 
to delve into both the theoretical and the practical background of the 
study. It is probable that with a clear understanding of the fundamental 
issues the researcher will pursue the study less mechanically, will interpret 
the data more accurately, and will make clearer the relationships of the 
individual study to other researches. 


4. Doctoral candidates need to differentiate the relative importance 
of various methods of research and the applications of them to particular 
kinds of data. For example, descriptive research, except as it may be used 
in certain historical studies, frequently does not contribute a great deal to 
the solving of basic problems. Survey type data rarely satisfy the test of 
any hypothesis. They do, of course, frequently suggest hypotheses for 
further study. Therefore the doctoral candidate might well regard norma- 
tive research as a first step toward solving a problem, relying subsequently 
upon the application of other methods of research for making a significant 
contribution. 


5. Researchers need to understand the relative value of new knowledge. 
Even from normative data new units or dimensions of a phenomenon 
may be obtained that have never before been known. Yet, even as new 
knowledge, these have little meaning unless they are related in some sig- 
nificant way to other data. Whether the relationships are utilitarian or 
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academic it is at the point that the validity and reliability of the relation. 
ships are established, that the researcher has made a contribution. 


6. In the future, doctoral candidates should give serious consideration z 


to the merits of longitudinal studies. 

7. The social scientist sometimes excuses himself from employing 
rigorous research techniques on the grounds that the prediction of human 
behavior and the precise measurement of human attributes cannot be made, 
The implications for the doctoral candidate are: (1) He must not be content 


until more of the unknown about human behavior has been studied. (2) As 
the researcher, he must not hide behind the cloak of unpredictability. § 
Upon the social scientist may lie the responsibility to develop and use new § 


techniques not known to the physical scientist but which are nonetheless 
sound. 


8. It would seem that had many of the researchers carried through a 
pilot study they would have gained considerable insight into the difficulties 
to be encountered and subsequently could have prepared themselves to 
cope with them. 

9. A number of the doctoral candidates sought the cooperation of 
educational institutions or commissions. Another approach is for the indi- 
vidual to ally himself with a research commission already engaged in a 
worth-while project and from his work with them carry out for his doctoral 
study a phase of the larger problem. The latter approach has the advantage 
of working within a larger design which has had the benefit of the thinking 
of persons experienced in research activities. 


Implications for Educational Institutions 


1. A lack in the coordination of the research efforts of doctoral candi- 
dates seems to remain one of the weaknesses of research in educational 
administration. Most of the dissertations reported in this study represented 
single, isolated pieces of research. It would seem that the institutions 
contributions in research would be much greater if they would develop 
long-range, over-all plans for research. This would involve careful selection 
of the more important problems, the development of a research design to 
encompass many individual studies and the encouragement of students 
to engage in the research. Such a plan might result in attracting outstand- 
ing candidates to an institution interested in pushing forward the research 
in a particular area. Furthermore, such a coordinated plan for research 


would tend to capitalize on a fuller background of experience within the 7 


institution whereas now it would appear that the energies of many 


researchers are expended without correspondingly significant results. 
2. The approaches to research in the social sciences have been patterned 


rather largely after those of the physical sciences. Although these | 
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approaches have tended to provide a degree of academic respectability, 
there is some question as to whether they have always yielded significant 
results. From this study it would appear that institutions should encourage 
the doctoral candidates to try new approaches in working with data. Admit- 
tedly first attempts might be rough and inadequate. Yet, one weakness 
now present could be avoided—namely, forcing into a traditional research 
mold, data which cannot be treated appropriately in the traditional manner. 
Such an implication for exploration in research methodology need not 
result in less rigorous attention to clear thinking, logical analysis, controlled 
conditions, and the testing of hypotheses. 

3. As a counterpart to one of the implications for researchers, institu- 
tions might very well encourage doctoral candidates to engage in longitudi- 
nal research. The institution, more than the individual, can provide 
oportunities to do longitudinal study under a plan of coordinated research 
by the university. 

4. Although it would probably be undesirable for an institution to 
adopt rigid regulations regarding doctoral research, the variation in the 
quality of studies at a given institution would suggest that faculties of an 
institution might well consider what constitutes acceptable research. 


5. Research in educational administration at the various institutions 
might be strengthened through (a) encouraging students to pursue advanced 
work in disciplines related to educational administration, (b) developing 
common research interests with faculty members in related disciplines, 
(c) attempting to evaluate cooperatively the dissertations from other disci- 
plines that bear on educational administration. 


6. Institutions within a state and across the nation have a responsi- 
bility for the dissemination of information about research achievements at 
their institutions. At least two types of information are needed: (a) a state- 
ment of research problems which are being studied; (b) a brief annotation 
of the researches completed by doctoral candidates. This proposal, if it 
is to be of most value, involves developing an effective, yet simple plan 
for collecting and disseminating the research regularly. 


Implications for Educational Administration 


1. The review of the doctoral studies would suggest that the practicing 
administrator has two important concerns with research in educational 
administration. First, he should feel a responsibility for helping to identify 
the significant problems that need study. Second, in light of providing 
better answers for his use in administering educational systems, he should 
be concerned with improving the quality of the research. In relation to 
both of these concerns is the need to establish a clearing house for doctoral 
research so it is readily available to the practicing administrator. 


57 





CALIFORNIA JOURNAL OF EDUCATIONAL RESEARCH Vol. VIII, No.2 


2. Research in educational administration has not been coordinated 
sufficiently to make it possible to develop from it a theory of administra- 
tion. The practicing administrator, of necessity, operates from day to day 
on information which has been determined largely on an a priori basis, 
A plan of coordinating and synthesizing research which would help in 
developing and integrating a theory of administration is needed badly, 
Undoubtedly, the time has come for a century-old profession to begin 
integrating its varying points of view into a systematic and dynamic theory, 


California Educational Research Association Annual Conference 


The thirty-fifth annual research conference of the California Educational 
Research Association will be held on March 15 and 16 at the Miramar 
Hotel in Santa Barbara. In an effort to make this year’s meeting truly 
statewide the Association is holding the 1957 conference in the southem 
half of the state. Last year they met in Fresno, and for a number of years 


the conferences have been held exclusively in the northern half of the state. 


The opening meeting will be held in the Conference Room of the 
Miramar Hotel at 8 p.m., Friday, March 15. Dr. James C. Stone of the 
Department of Education, University of California at Berkeley, will speak 
on “The University of California Experiment in Teacher Education.” Two 
other general sessions will be held on Saturday. There will be five section 
meetings running concurrently throughout the morning and the afternoon 
of the second day in which research workers will report upon the methods 
and results of their newest research projects related to schools and educa- 
tion. Reports will be presented by Some of California’s outstanding 
educational research workers, including representatives of city and county 
school systems, universities, colleges and junior colleges, and the State 
Department of Education. 


Membership in the Association is open to all who desire to support 
educational research in California. The dues are two dollars a year which 
may be paid to the secretary-treasurer at the conference. The 1956-57 
officers are: President, Edward Taylor, Alameda County Schools Office; 
First Vice President, Jack Holmes, University of California, Berkeley; 
Second Vice President, Hazel Lewis, Stockton City Schools; and Secretary- 
Treasurer, Glenn W. Durflinger, University of California, Santa Barbara 
College, Goleta, California.° 
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The Local Research Program 
and the Research of Practitioners 


STEPHEN M. Corey 


During the next forty-five minutes I am going to try to do three things. 
First, I want to make some statements about educational research and the 
process of educational improvement that I believe to be true and which 
have helped me in my work with practitioners. Secondly, I want to describe 
what appears to me to be the process of change that results in improvement 
of professional behavior. Finally, I want to relate this process of change 
to things that might be done in a local research program to increase the 
assurance that educational changes will represent educational improvements. 


Research and Improvement of Practice 


My first belief is that the chief purpose of educational research is to 
improve educational practices. Education is effective depending upon the 
practices of those responsible for and those participating in the educational 
process. Knowledge, understanding, attitudes, feelings and sentiments 
about education are all important, but practices, and their consequences, 
ultimately constitute education. At least this point of view, as I have said, 
has helped me. It means, of course, that educational research is predomi- 
nantly and unashamedly practical. It is, in some ways, analogous to 
medical research which is practical and is undertaken to improve what the 
medical practitioner does. Research in psychology, or psychometrics, 
history, sociology, economics or philosophy is related, in my judgment, to 
educational research in somewhat the same way as research in physiology, 
biochemistry or bacteriology is related to medical research. 

A second belief or assumption that has helped me goes like this: 
Improvements in education are primarily a result of changes in the way 
people act. In other words, the betterment of educational practices requires 





Stephen Maxwell Corey is Dean of Teachers College, Columbia University. 
He has served in numerous professorial capacities before coming to his present 
position, particularly at Teachers College, the University of Chicago, and the 
University of Wisconsin. He is a member of several learned societies and the 
author of many books and articles, including “Action Research to Improve School 
Practices,” which was published by Teachers College, Columbia University in 1953. 
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that practitioners change what they do. This seems to me to be almost 
self-evident and suggests that the educational investigator, the educational 
researcher must be, in addition to everything else, a student of the process 
of change. 


A third belief that has been useful to me is that people are more 
disposed to change, and more likely to improve—which is more than 
change—in groups than as individuals. There are many reasons, I presume, 
why this might be so. One important reason is the pressure exerted upon 
all of us by the norms and expectations of the groups to which we belong 


willfully and pridefully. Another reason is that the resources within a 


group, in some contrast to the resources of an individual, give added 
assurance that any problem will be dealt with more realistically. 

Finally, I believe that the method of research, a procedure which 
requires increasingly rigorous and disciplined inquiry and experimentation, 


is an exceedingly promising way for groups of professional practitioners to f 
improve what they do. I see this approach to improvement as being quite f 


different from that which places major dependence upon authority or 
tradition. 


These beliefs, and my experiences related to them, have caused me to : 


have great confidence in research conducted by groups of practitioners— 
that is, by teachers, principals, superintendents and supervisors—in order 


for them to improve their professional work. By research I mean, increas- } 


ingly careful and objective attention to the following elements in the 


process of inquiry: 1. Definition of problems; 2. Search for promising 


solutions; 3. Design of tests of these promising solutions; 4. Appraisal of 
the effects of these tests; and 5. Generalization to identify tentative guides 
for future practice. 


For practitioners to become increasingly careful and objective and 


thoughtful in these respects is, however, not easy. Anyone who has tried F 


it realizes that it is difficult. Most of us, under the pressure of our day by 
day work, leap from a vaguely defined difficulty into action to cope with 
it. Only infrequently, too, do we assess the effects of our action carefully 
to identify lessons that will help us in the future. In order to improve the 
quality of our problem solving, we need help which, speaking generally, 
has not been given us. 


The fact that practitioners need help to improve the quality of the 
research they undertake to better their practices has, for me, clear implica- 
tions for an effective local research program. I wish to develop these 
implications and do little else during my remaining time. Concentrating, 
thus, on the local research program’s contribution to the training of practi- 
tioner-researchers is not meant to imply that the program should be 


concerned with nothing else. In my judgment, however, no other function | 


of a local research program’is comparably promising or important. 
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The only assumption I need make about a local program in order to 
develop my thesis is that one or more people have defined responsibilities 
in connection with it. There is, in other words, some professional personnel. 
The central problem then is, what can this professional personnel do to 
facilitate the work of teachers, administrators and curriculum leaders who 
are trying to use research as a method of self improvement. 


The Process of Self Improvement 


I should like to turn now to a consideration of the process of self 
improvement. In common, I presume, with most of you, I have tried hard 
to understand somewhat better what this process is like. To the degree 
that I can comprehend it, as it applies to myself and to others, I seem to 
be better able to exploit research procedures as a methodology of educa- 
tional improvement. 

1 wish we knew more about the process of self improvement and the 
factors that influence it. Some people seem to be reluctant to experiment 
with their professional behavior, to try out new things. They go on doing 
what they have been doing. Others seem to find experimentation and 
change exciting and attractive. My hypothesis is that this kind of change- 
ability is different from what we call native intelligence, and that it is a 
pervasive, temperamental disposition which manifests itself to varying 
degrees in particular fields of activity. I have friends, for example, who 
are very experimental in relation to food. They seek opportunities to try 
out different national dishes and are adventurous and exploratory in their 
own home cooking practices. Other friends of mine have eaten mashed 
potatoes, gravy, meat and pie all of their lives and intend to do so for the 
foreseeable future. Some people are experimental in their clothing—and 
I am not confusing this kind of experimentation with the rapid changes in 
dress dictated by fashion. I, personally, am quite experimental in my 
eating but find myself shamefully rigid so far as clothing is concerned. 

Some day it may be possible to predict the disposition to change of 
prospective teachers, principals, curriculum coordinators and superintend- 
ents. The information could be used most advantageously in selecting 
people for admission to teacher education institutions or for professional 
positions in education. The resistance that some of us put up to change 
is so great as to mean that trying to train us is almost a waste of time and 
money. A teacher education program that involves students or teachers 
in service who are, on the other hand, disposed to change and to experiment 
would, almost inevitably, be effective. Similarly, if we were to appoint 
to leadership positions in education only those persons who not only repre- 
sented advanced practice but also were disposed to modify and change 
their practices, the gap between what education is and what it ought to 
be would be appreciably reduced. 
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In my effort to learn more about the process of change as it is experi- 
enced by educational practitioners, I constantly ask people who are proud 
of some innovation in their professional behavior this question: “How did 
you come to do it?” “What is the process that led you to stop, for example, 
planning every single item of your high school staff meeting agenda and 
begin to involve the entire teaching faculty in the development of plans?” 


This question, incidentally, is hard for most of us to answer. Our 
training, our education, has taught us to examine critically what we read 
and, sometimes, what we see. We have had little practice, however, 
examining our own experience to determine why we did something. Hard 
though it may be to do so, enough people have been able to comment on 
their experience as they changed to lead me to the tentative conclusion 
that the process goes something like this. 

First, all of us are continuously appraising the effects of our professional 
activities. This we cannot avoid because we are, it appears, inveterate 
evaluators. This continuous running appraisal of the consequences of our 
professional behavior involves an examination of its relationship to our 
aspirations. Most of the time, our perception of the effects of our behavior 
does not disturb us particularly because it does not seem to be greatly out 
of line with what we aspire to do. When this is the case, we do not, of 
course, try to change, to improve. 

But in the lives of all of us, there come times when we are upset, made 
uneasy, thrown out of equilibrium, by observing the effects of our profes- 
sional behavior that are in sharp contrast with what we want. We may, 
of course, try to deny that this is our fault, but if the dissatisfaction persists, 
we decide that we must do differently, that we must change, that we must 
improve. This, in my judgment, is not only the first but the most important 
single aspect of the change process. 

Just being upset, however, does not necessarily lead to change, as 
everyone knows. I have been vaguely uneasy about certain aspects of my 
own professional behavior for months, but the feeling was not strong 
enough to lead to the second phase of the change process. This second 
phase involves a much more careful assessment of the situation out of 
which the uneasiness grows. This more careful assessment is undertaken 
to determine, more specifically, just why our achievements, as we see 
them, and our aspirations are so different. This attempting to “define the 
problem,” as we say, can be awkward and fumbling or skillful and effective. 

After we see more clearly what difficulty we must cope with, we move 
into a more or less thorough search for practices that give promise of 
improving the situation that displeases us. This seems to me to be the 
third phase of the change process. To the degree that this search for more 
promising practices is undertaken with a research orientation, the process 
becomes complicated and difficult. It involves more than the exploitation 
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of numerous resources that might suggest these promising innovations. It 
also involves a critical examination of their use elsewhere and an imagina- 
tive projection of their use in a local and unique situation. This latter is 
essential in order to select among numerous possible practices a more 
limited number to be tested in an on-going, real situation. 

After the strong feelings of discontent, and the more careful definition 
of what seems to be wrong, and the identification of something promising 
to try as a remedy, comes the development of a design for the “try out” 
that will make it possible to know, as realistically as possible, what the 
effects of the presumably better practices are. This question of experi- 
mental design is a troublesome one in the research of practitioners. So 
often a proposed plan is received with great enthusiasm, and an agreement 
reached “to try it out,” but what “it” is is vaguely understood as is a method 
of testing that will reveal whatever advantages the new practice may have. 
Frequently, too, what is alleged to be experimentation is carried on under 
circumstances that make it impossible to know whether or not subsequent 
effects are a consequence of the sheer passage of time, or the enthusiasm 
of the teacher, or the generally increased interest of boys and girls in strange 
and different kinds of work. 

The final aspect of the change process is the reassessment of the total 
situation to determine whether or not the effects of the change are consistent 
with the good predictions. This kind of final assessment or evaluation, 
again, is often superficial and subjective. It is, of course, impossible to do 
an effective job of appraisal on the sudden inspiration, “it is time to get 
evaluation data.” The final appraisal must be part and parcel of the total 
experimental design. 

So much for a description of the change process as it seems to me to 
take place. The key words are dissatisfaction, assessment, search, design 
and appraisal. In the great majority of cases we are, of course, but vaguely, 
if at all, aware of these different phases as we move through them. Their 
sequence, too, is rarely as neat and obvious as my analysis has suggested. 
I do believe, however, that whenever change occurs—and by that I mean 
the kind of change that is consciously undertaken in order to improve 
professional practices—it does illustrate at one time or another each of 
these change phases. 





What the Local Research Program Can Do 


With this description before us I want to turn now to what the local 
research program might do to improve the quality of the change process. 
We can only have confidence that change is improvement to the degree 
that we are careful and thorough and objective as we: a. assess the instruc- 
tional situation that disturbs us, getting as much evidence as we can to 
find out the relationship between our present practices and our intentions; 
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b. use many resources—books, articles, visits, the ideas of our associates— 
for suggestions about possible improvements in our behavior; c. introduce 
innovations in such a way as to know not only what it is that we are doing 
but also the effects of what we do, and finally; d. generalize from these 
effects to maximize our learning gains. 


Let’s look for a moment at the first phase of the change process. What 
can the local research program do to increase the likelihood that practi- 
tioners will continuously examine their aspirations and will get increasingly 
reliable and valuable data on the effects of their practices in relation to 
these aspirations. None of us knows a great deal about what research 
consultants can do to help in these respects. The practice that is tried 
most frequently is probably least promising. This practice involves the 
outsider, the consultant, telling the practitioner, maybe with some subtlety, 
that what he is doing needs improvement. A much more promising prac- 
tice is for the consultant to do whatever he can to make it possible for 
practitioners to come together and talk about their work under circum- 
stances that encourage candor and self scrutiny. 


In my attempts to help people examine their purposes, and the relation 
between these purposes and their achievements I have not found the 
typical group problem census very helpful. Too often problems are identi- 


fied about which there is no strong disposition to do anything. I have 
had more success encouraging discussions in which group members state 
what they would like very much to do but as yet have not been able to. 
This more positive and affirmative approach gets around to problems and 
difficulties in due course but not right at the start. 


As I said a moment ago, aside from therapeutic situations, relatively 
little is known about things that may be done to encourage and help 
practitioners examine what they are doing and their aspirations in order 
to identify the kind of inconsistency that leads to change. This would be 
a good area in which personnel in local research programs might do some 
experimenting—some action research—of its own. 


When the practitioner tries to get more precisely at the nature of his 
difficulty, the local research consultant, if he is wise, can be helpful. Few 
. practices help more than a continuous encouragement to “get some data.” 
This is the kind of thing that Hilda Taba and the late Ruth Cunninghan, 
among others, have stressed with such great success in their work with 
teachers. In our work in the Horace Mann-Lincoln Institute of School 
Experimentation we have found that many practitioners talk quite vaguely, 
though often vigorously, about their difficulties. Most of us, as a matter 
of fact, converse about our own work difficulties without much discipline 
or any clear conception of a somewhat scientific method of problem solving. 
We confuse opinions with facts, speculations with conclusions; and our 
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motivation to think and talk in quantitative terms is very weak. One of 
the first things we try to do in our Institute group work is to establish some 
norms that place value upon differentiating between opinion and fact and 
upon references to data, to evidence, to information. Unless something 
of this sort is done, we frequently either define out of the total troublesome 
situation problems we subsequently find we do not want to work on or we 
do go to work on what turns out to be the wrong problem. 


This latter situation was illustrated in one of the Institute’s projects 
which I have already described to some of you. A small group of primary 
grade social studies teachers had experimented for 18 months with new 
methods and materials and the data they procured on the effects of their 
experimentation were most encouraging. A long staff meeting was held 
during which the experimenting teachers reported their experience and 
urged that all other teachers in the same grades adapt and use the same 
methods and materials. There was considerable enthusiasm for this. When 
the chairman put the question, all of the non-participant teachers said 
they wanted to do it, too. 


As time went on, however, it became clear that these teachers that 
had not participated in the experimentation were not changing their prac- 
tices. This led to considerable uneasiness on the part of some of the 
leadership and an attempt to find out what “was really wrong.” In inter- 
views with teachers, the comment was made again and again that these 
new methods took too much time and that the teachers were so burdened 
with routine activities that they could not get around to doing what they 
really wanted to do. 


This having been identified as the problem, a great deal of trouble 
was gone to in order to relieve teachers from some of the onerous details 
they said were consuming their time—like paper drives. Considerable 
success was met in reducing the routine tasks of the teachers, but, such is 
this world’s sadness, no increased experimentation with the social studies 
program was visible. 

This led to a great deal of concern and a much more careful inquiry 
to determine what was wrong. This inquiry had to be careful because 
what actually was wrong was not that the teachers didn’t have time. They 
didn’t know what to do. It is much easier to say you don’t have time than 
it is to say you don’t know what to do. When it did become clear that the 
lack of spread of these demonstrated practices was a consequence primarily 
of the fact that the teachers had never learned how to engage in them, 
in-service training was undertaken which proved to be quite successful. 

The local research consultant is quite apt to be trapped himself at the 
problem definition stage. He finds himself in this embarrassing situation 
less frequently, however, to the degree that he keeps pushing, gently and 
with circumspection, for evidence about the problem. 
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After some agreement has been reached by the researching practitioners 
as to the nature of the difficulty they must deal with, they begin their 
search for what to do about it. It is at this stage, it seems to me, that the 
personnel in the local research program can be of benefit by helping the 
practitioners to learn about the relevant research and enlightened specula- 
tion that are already available. A great deal of operational or action 
research is disappointing because it is repetitious and does not involve 
taking advantage of what already is known. It is important, however, for 
the research consultant to recognize that it is not easy to take advantage 
of what is known. To assume that because an expert knows something 
and tells the practitioner, the practitioner can take advantage of what he 
is told, is a snare and a delusion. Research consultants need much more 
ingenuity than most of us have if we are to help practitioners become 
acquainted with and perceive the pertinence of the accumulated experience 
of others who have tried to deal with similar problems. 





I recall working with a group of teachers who were convinced that an 
emphasis in American history classes upon the biographical method would 
result in improvements in the character of their pupils. During a rather 
long period of time when this belief was being discussed and prior to the 
design of any test of it, I talked a lot about research that had been con- 
ducted to determine the effect of direct moral instruction upon behavior. 
The classic studies by Hartshorne and May, for example, were cited and | 
analyzed but, either because of lack of consultant skill, or because of some | 
absence of readiness, all of this body of pertinent literature and informa- | 
tion dealing with analogous situations was just overlooked by the 
experimenters. 





The next phase of the practitioner research process, designing the 
experiment or designing the trial, calls for considerable technical skill. 
This is a kind of skill, too, that people who have leadership responsibility 
in the local research program are quite apt to have. Much of the graduate 
training intended to make us better educational investigators has concen- 
trated upon design problems. I have found that one of the most effective 
things that I seem to be able to do as a research consultant to practitioner 
investigators is to try to make design and its importance highly visible. 
This is almost always initiated by trying to get a verbalized step by step 
description of what is to be done and why. 


In training situations at Teachers College where we try to help cur- 
riculum people become more systematic in their problem solving procedures, 
a great deal of actual practice is provided in the description of experimen- 
tal designs for testing certain practical hypotheses. The temptation, of 
course, when attention is being given to design by the highly skilled investi- 
gator is for him to be so impressed by the necessity for controls that he 
gradually abstracts the research situation clear out of this living world. 
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Frequently in the kind of research I am talking about, niceties of design 
have to be sacrificed because of practical exigencies. Again, though, 
granting these difficulties, the research consultant who is aware of design 
problems and can communicate what he knows, helps the experimenting 
practitioner greatly. 

Finally, of course, after the innovation has been in effect for a period, 
data regarding its effectiveness must be accumulated, and these data inter- 
preted. Here the disposition of most of us practitioners is to take data at 
their face value and be almost completely oblivious of problems of sampling 
or reliability or validity. These problems, however, are not beyond the 
ken of intelligent people. Teaching about them can be direct and effective 
whenever people realize that what they learn will help them improve their 
professional behavior. In relation to procuring evidence, and to measure- 
ment and statistics the trained investigator or educational researcher has 
accumulated a great deal of skill and information. Communicating what 
he knows, however, is not at all easy. 

Sometimes what appears to be a casual conversation can result in an 
appreciably greater attention to getting data on effects. Here is an 
illustration: 


Teacher A: It was wonderful, just wonderful! He recited all 30 verses 
without a mistake. And with so much feeling! 

Teacher B: You feel real good about those youngsters reciting poetry, 
don’t you? 

Teacher A: I certainly do. They learn so much. 

Teacher B: I suspect you’re right. What kind of things do you see them 
learning, for example? 

Teacher A: Oh, to love good poetry for one thing. 

Teacher B: Well, that’s certainly worth learning. 

Teacher A: I think so, too. 

Teacher B: I wish I were sure of a good way of finding out whether or 
not ninth graders were improving their taste in poetry. I'd 
like to push them off the limerick level and be sure they 
stayed off. 

Teacher A: Oh, I guess I can just tell when they are enjoying it. At 
least, I’m pretty sure I can. 

Teacher B: It would be interesting to find out if they read a bit of good 
poetry on their own, wouldn’t it? 

Teacher A: Yes, it would. They say they do. Or many of them say that 
anyway. How else could I find out? 

Teacher B: Oh, I’m not sure. It might be interesting to take a look at 
what they choose during free reading periods. 

Teacher A: That wouldn’t be too hard, would it? 

Teacher B: No. Another thing might be to have them finish an incom- 
plete story about a ninth grader who, say, “strolled into the 
library, his studying finished, and went directly to the shelf 
for .. .”. Some good book on evaluation probably could 
help here. 

Teacher A: You know, I might do that unfinished story business at the 
beginning and end of the semester. 
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Conclusion 


During the past ten or fifteen minutes, I have tried to indicate, by 
concentrating on the process of change, some of the responsibilities of the 
local research program if it is to help practitioners conduct inquiries and 
experiments with increasing rigor and objectivity and thoughtfulness. As 
I have said several times, the reason for this increased rigor and objectivity 
and thoughtfulness is to increase the likelihood that changes in practice 
that grow out of experimentation will represent actual improvement. 


In our work in the Horace Mann-Lincoln Institute, we have found that 
anyone who hopes to be of much benefit to practitioners who are conduct- 
ing research to improve their practices must develop special competence 
along at least two dimensions. First he must be a skillful teacher of those 
group processes that enable people to communicate better with one another 
to the end that they can muster as many as possible of their resources to 
deal with practical educational problems. The second dimension has to 
do with the methodology this group employs in its problem solving. We 
have learned, too, that we must first give major attention to building a 
group and subsequently concentrate on research methodology. 


In the university training most of us have had as educational researchers, 
relatively little attention was paid to the conditions that must obtain if 
groups of practitioners are to improve their research methods. Relatively 
little attention was paid, too, to what needs to be done in order to teach 
practitioners to conduct research. It seems to be assumed that anyone 
who has himself learned how to do research can teach others. I suspect 
that there is no more relationship between an individual's ability to conduct 
good research and his ability to teach others how to do it, than there is 
between the ability of a person to play a good game of tennis and his 
ability to teach someone else to play the game. 

In conclusion, I would like to recommend that those of you who are 
interested in what can be done to encourage practitioner research, be sure 
to get the forthcoming A.S.C.D. Yearbook entitled Curriculum Research. 
A. W. Foshay and James Hall were co-chairmen of the production com- 
mittee. Among the other authors are Margaret McKim, Hilda Taba, John 
T. Robinson, Virgil E. Herrick, Matthew Miles, and Ronald Doll, all of 
whom had extensive experience with cooperative field research, and they 
write about it thoughtfully and lucidly. 

I would also like to call to your attention a little book a group of us 
in the Horace Mann-Lincoln Institute of School Experimentation wrote 
entitled Training Curriculum Leaders for Cooperative Research. This 
booklet describes a 14-month program of leadership training and, if it does 
nothing else, calls attention to a few practices that may not be worth 
repeating. 
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Physical Maturing Among Boys 
as Related to Adult Social Behavior 


A LONGITUDINAL STUDY 


RoBert AMES 


The influence of rate maturation upon the social behavior of adolescent 
boys has been investigated by Jones and Bayley (7), who found consistent 
differences in favor of early-maturing boys, as measured by skeletal age 
indices from X-rays, on such variables as popularity and attractiveness, as 
measured sociometrically by an opinion test. The present study is con- 
cerned, not only with the relationship of maturation rate to adolescent social 
behavior, but also with the relationship of this variable to the social behavior 
of males in adult life. 


This study is part of a larger longitudinal investigation of 40 males and 
40 females in which an attempt was made to determine the long-term 
predictive value, or consistency, of measurements of social participation 
and leadership which were obtained during adolescence (1). Rate of 
maturation during adolescence was considered, at the beginning of the 
original study, to be simply one of the “additional variables” which, among 
other antecedents, should be examined with regard to a possible contribut- 
ing influence. It soon became apparent, however, that for the men of the 
sample, physical maturation rate during adolescence was more highly 
related to adult social behavior than were the various measures of adolescent 
social behavior. A complete report of the original study will be made 
elsewhere, but it was felt by the investigator that the effects of maturation 
would be of interest to educators. 


Robert Ames is Assistant Professor of Education at Los Angeles State College 
of Applied Arts and Sciences. Before obtaining his present position in 1956, he 
was Consultant in Educational Psychology (1954-1955) and Research Fellow, U. S. 
Public Health Service, Institute of Mental Health (1955-1956), at the University 
of California at Berkeley. He obtained his doctorate from the University in 1956. 
This article is a digest of his dissertation, which was done at the Institute of Child 
Welfare, University of California at Berkeley, under the sponsorship of the U. S. 
Public Health Service. 
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Data and Procedure 


The subjects of the present report were 40 men who were members 
of the Adolescent Growth Study of the Institute of Child Welfare at the 
University of California, Berkeley, California. The Adolescent Growth 
Study is a longitudinal investigation of the social, intellectual, and physical 
development of a group of boys and girls, originally over 200 in number, 
who were studied between the years 1932 and 1938, when they were 
between the ages of 11 to 18 years (4, 5, 6). The present study is part of 
a larger follow-up of these same subjects at an average age of approxi- 
mately 33 years. 


The roentgenographic technique for determining physical maturity in 
terms of skeletal age is described elsewhere by Bayley (2). This procedure, 
which involves taking X-rays of the hands and knees at a standard distance, 
gives a highly reliable index of skeletal development by means of epiphyseal 
closure, appearance of carpals and amounts of ossification. Skeletal age 
indices obtained during the preadolescent and adolescent years were avail- 
able for the subjects of this study. Scores of skeletal age divided by chrono- 
logical age during the 14th to 17th years of life were averaged for each 
individual in order to obtain an index of his relative rate of maturation. The 
resulting ratios were then converted to standard scores. A high score indi- 
cates early skeletal maturation. 

Ratings of the adolescent social behavior of these subjects were available 
in the form of standard scores. Two types of observer-ratings were obtained, 
the “ICW” ratings and the “Clubhouse” ratings. 

The ICW ratings of “sociability” (among other traits) were obtained 
during one-hour observations of these subjects in same-sex, free-play, small 
group situations. Independent observations were made by at least three 
adult observers (8). 

The Clubhouse “popularity” ratings were made over longer time inter- 
vals and for a much wider variety of behavior. Subjects were observed in 
both same-sex and opposite-sex situations, in small groups and in large 
groups, during organized activities and during informal gatherings. 

Opinion Test “popular” ratings are peer-ratings in the form of an 
opinion test patterned after the “Guess Who” test of Hartshorne and 
May (3). This sociometric device was employed to obtain the adolescents’ 
judgments of one another in terms of social behavior. A description of 
the procedure is reported by Tryon (9). 

Many other variables for both sexes, such as intelligence test scores, 
years of education, parental socioeconomic status, and adolescent measures 
of leadership, height, weight, and strength, are investigated in the larger 
study of which this report is a part, but for the purpose of brevity these 
will be reported upon elsewhere. 
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Direct interviews were used as a part of the follow-up procedure. The 
interviews were standardized, semi-structured and open-ended. Answers 
to the interview questions were rated on seven-point scales independently 
by two-raters. Raw scores obtained by this procedure were combined and 
converted to standard scores. 

Three types of adult social participation were considered to be of value 
for investigation. These were “informal social participation,” “formal social 
participation,” and “occupational social participation.” 

Informal social participation was defined and rated in terms of the 
frequency with which the subjects entertained, or were entertained by, 
friends, relatives, casual acquaintances, etc. Also included in this category 
were various sports and hobbies more or less socially oriented in nature. 

Active membership in clubs, societies, and other structured organizations 
was considered to be an index of formal social participation. Mere nominal 
membership was given no weight in the ratings. 

Occupational social participation was quantified by rating the occupa- 
tions of the men in terms of the relative amount of social contact which was 
afforded by them. Judgments were aided by supplementary statements 
from the subjects as to the amount of social participation involved in their 
occupations. 

Adult leadership was defined in terms of occupational status and 
organizational officership. The Warner Scale of occupational status was 
used, supplemented by ratings of the interview protocols in terms of the 
subject’s evident position within his particular occupation. Officership of 
formally organized activities was found not to be a continuously distributed 
variable in this sample. The individuals either held some position in an 
organization, or they did not. Organizational leadership was therefore 
treated operationally as a dichotomous variable. 











TABLE | 
Rater Reliabilities 
OO Correlations (r) Standard Error 
Between Ratings 4 ofr 
Informal Social Participation 83 05 
Formal Social Participation 93 02 
Occupational Social Participation 91 .03 
Occupational Status 83 05 





4 Ratings of responses to interview items were made independently by two raters. 


Results 


The independent ratings made by two judges of the interview protocols 
are substantially high in agreement, as may be observed in Table 1. 
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Pearson product-moment correlations were computed by the well-known 
machine formula. The 28 1’s resulting from intercollerating eight variables 
are shown in Table II. 


TABLE II 
Intercorrelations of Selected and Adolescent Measures 


No. No. No. No. No. No. No. 
Variables 1 2 3 4 5 6 7 
Adult Measures 
No.1 Occupational Status 402 22 51b 10 09 14 
No.2 Informal Social Participation ‘ 392 26 ©6392 = 23 
No.3 Formal Social Participation 16 13 «6.00 ~-.05 
No.4 Occupational Social 
Participation -04 26 14 
Adolescent Measures 
No.5 ICW “Sociability” 
No.6 Clubhouse “Popularity” 
No.7 Opinion Test “Popular” 
No.8 Maturation Rate (Skeletal) 








a Significant at the 5 per cent level. 
b Significant at the 1 per cent level. 


Reference to Table II, Row 8, Columns 5 and 7, indicates that skeletal 
maturation rate is related to the adolescent measures of ICW “Sociability” 
and Opinion Test “Popular” at the 5 per cent level of confidence. These 
relationships tend to confirm the findings of Jones and Bayley, mentioned 
earlier. 


Of more importance for long-term predictive purposes, however, are 
the correlations shown in Rows 5, 6, 7, and 8, Columns 1, 2, 3, and 4. 
These correlations indicate that skeletal maturation rate is more highly 
related to each of the adult measures of social behavior than are any of 
the adolescent measures of social behavior. 


Clubhouse “Popularity” is the only measure of adolescent social behavior 
which correlates significantly with any of the adult ratings. The correlation 
of .39 with Informal Social Participation is significant at the 5 per cent level 
and is reasonably linear in form. The r of .48 between Maturation Rate 
and Informal Social Participation, however, is significant at the 1 per cent 
level. 


None of the adolescent ratings of social behavior correlates significantly 
with the adult measures of Formal Social Participation or Occupational 
Social Participation. Skeletal Maturation Rate, however, is significantly 
related to both of these adult measures. 
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The ratings of the three types of adult social participation were com- 
bined in order to obtain scores of Total Participation. The r between 
Total Social Participation and Clubsouse “Popularity” is only .32. None of 
the other ratings of adolescent social behavior is signficantly related to 
Total Social Participation. Maturation Rate, however, correlates .52 with 
Total Social Participation. 


The group was divided into three categories to be called, respectively, 
“early maturers” (Maturation Rate above a standard score of 50), “average 
maturers” (Maturation Rate 50), and “late maturers” (Maturation Rate 
below 50. It is noteworthy that only one late maturer is above the mean 
of Total Social Participation. 


Fifteen of the males of the sample were found to be in occupational 
positions which require that they supervise or direct the work of subordi- 
nates. Of these fifteen subjects, only three were late maturers. A biserial 
r of the two groups (directors and non-directors) with Maturation Rate 
yields a coefficient of .51, which is significant at the 1 per cent level. 


Only eight of the men in this study hold offices in any type of social 
organization, club, lodge, or civic group. Not one of these “organizational 
leaders” was a late maturer. The mean maturation rate of the leaders was 
58.13. The mean maturation rate of the non-leaders was 47.9. Because of 
the small number of organizational leaders found in this sample, a statistical 
analysis of these differences would be psychologically meaningless, but the 
fact that these differences exist is consistent with the other findings of this 
study. 


Discussion 


There appears to be little doubt that, for the males in this sample, rate 
of maturation during adolescence, as measured by skeletal age indices 
determined from X-rays, is a better predictor of the adult social behavior 
herein defined than is any other variable included in the present study. If 
the investigator were to accept only the 1 per cent level of confidence as 
a test of the null hypothesis, none of the ratings of social behavior made 
during adolescence could be considered to be significantly related to any 
of the adult measures of social behavior. Skeletal maturation rate, how- 
ever, would remain significantly related to occupational status, informal 
social participation, formal social participation, total social participation, 
the supervision of subordinates in occupations, and possibly to officership 
in organizations. 

The early-maturing boy is recognized by his peers and by adult observ- 
ers as being socially active and popular. It is highly probable that, with 
his physical superiority and his earlier acqpuisition of more adult drives 
and attitudes, he acquires patterns of success with his peers which tend 
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to persist over time. It is hypothesized by this investigator that early 
maturation for boys is accompanied by physical, psychological, and social 
advantages which remain rather stable. Early maturers tend to either 
become or remain socially active in adulthood, whereas late maturers tend 
to either remain or become less socially active as a group. 


The results of this study, and of the larger study from which it is taken, 
indicate that the stress placed upon social participation in the school may 
be somewhat overemphasized in terms of any value for future social 
behavior. The sociability and popularity of adolescent boys has, at best, a 
very low relationship with their social participation in adult life. Any 
“transfer” value of social behavior in the school is found, for males, to lie 
only in the area of informal social participation (particularly in socially 
oriented sports and hobbies) in adulthood. This is not to state that social 
participation should not be encouraged or supported in the schools. Such 
experience may well have some value for those who are successful in their 
social relationships during this.period. But those males who are successful 
at both the adolescent and adult level are found, in this study, to be the 
early-maturers. Unless some measures can be taken to assure continuing 
successful social experiences for the less fortunate members of the school 
society, it is highly doubtful whether social activities programs in the 
schools will be of much measurable value to them in later life. 


Several factors limit the general applicability of this study. The sample 
used is necessarily of a somewhat select nature in terms of socioeconomic 
status, intelligence, years of education, and geographical location. The 
sample is perhaps also select in the very fact of its availability for a 15-year 
follow-up. 


Ideally, the ratings of adult social behavior should have been made 
by direct observation in order to be more nearly comparable to the ado- 
lescent measures. Practically, this procedure was of course impossible. 
The investigator had to depend upon behavioral self-report from the sub- 
jects, supplemented where possible by verification. Further research should 
of course include another follow-up of these same subjects in another 
fifteen years, when they have presumably reached somewhat of a “peak” 
socially and occupationally. The small N is a definite statistical limitation 
of the present study. The findings of this study should be cross-validated 
with larger groups of subjects from differently selected populations. 


Of particular interest to this investigator were the relationships found 
between skeletal maturation rate and the social variables for the males of 
the sample. The consistency with which maturation rate relates to both 
the adolescent and the adult measures is noteworthy. Jones and Bayley (7) 
have previously pointed out the clear evidence of the effect of physical 
maturation upon the behavior of adolescent boys, and the reliability and 
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stability of skeletal age indices as measures of maturation. Further evidence 
would be desirable, in view of the findings of the present study, on the 
long-term effects of this essentially physical variable. 
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Directory for Exceptional Children, 1956, edited by E. Nelson Hayes 
Boston: Porter Sargent Publisher, 1956. 247 pages. $4.00 


This directory contains descriptions of more than 1800 public and pri- 
vate schools, clinics, homes and other facilities for the mentally retarded, 
cerebral palsied, orthopedically handicapped, emotionally disturbed, so- 
cially maladjusted, blind, deaf, cardiac, and epileptic. Included also are 
listings of psychiatric and guidance clinics, speech and hearing clinics, 
state programs and personnel, and associations, as well as an extensive 
index. 


This is the only reference work available to the rapidly growing field 
of special education, and has been compiled after much correspondence 
with thousands of teachers, school administrators, doctors, psychiatrists, so- 
cial workers, and others. It is a book which should be widely known as 
an invaluable help to parents and others seeking services for the more 
than three and one-half million children who are handicapped. 
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A Longitudinal Study of Learning 
by Medical Students 


PETER G. LoRET AND RICHARD B. WEST 


In evaluating the achievement of medical students throughout the 
United States in the subject matter of cancer, the Education Project of the 
Cancer Research Institute of the University of California has obtained data 
relative to their knowledge of cancer at different levels of their medical 
education. Since the “Examination for Students of Medicine in the Subject 
Matter of Cancer” has been administered annually to voluntarily partici- 
pating schools since 1949, it is possible to obtain test scores for students 
for each of the four years of their medical education. 

This investigation is an attempt to determine the nature of cancer 
learning in terms of gains in knowledge, as well as the rate and nature of 
these gains. It is based on an analysis of the test performance of a randomly 
selected, representative sample of medical students who have taken the 
Cancer Test annually for a consecutive four-year period. 


Methodology 


The Sample. Using Neyman’s method for obtaining a sample repre- 
sentative of all participating schools (1:459-461), a sample of four hundred 
students was randomly selected from a group (N=2,341) who had taken 
the Cancer Test as Freshmen in 1951 and in each of the three subsequent 
years of their medical education. 

In order to determine at each academic level whether the sample of 
four hundred was truly representative of the population from which it 
had been drawn, the null hypothesis that no significant difference existed 
between the means of the population and sample was tested. The t-test 


The co-authors of the present article are members of the staff of the Cancer 
Research Institute, University of California Medical School, San Francisco. Peter 
G. Loret is Assistant Research Oncologist (Medical Education) and Lecturer in 
Educational Psychology, School of Nursing. Richard B. West is Statistician. Dr. 
Loret’s Ph.D. was obtained at the University of California, Berkeley, in 1956. 
Mr. West is a graduate student at the University of California, Berkeley, and 
earned his Master’s degree at Pacific Union College, Angwin, California. This 
study was carried out in the Cancer Research Institute by the Cancer Teaching 
Project which is supported by Grants CS-1029 and CT-493 from the National Cancer 
Institute, United States Public Health Service. 
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indicated no significant differences between the means at any academic 
level, and the null hypothesis was therefore accepted. 


Analysis of the Data. To obtain a sample of performance which 
would be comparable throughout the years under consideration, only items 
which remained completely unchanged, or which involved only minor 
revisions, such as punctuation, spelling, and the like were retained for the 
present analysis. The number of items thus obtained was sixty. The 
number of correct responses to each of these items was counted for the 
four hundred students for each year (Freshman through Senior). These 
data were converted to per cent correct for each item at each academic 
level. To facilitate the analysis of these data, the items were then separated 
into groups by site of the neoplasm involved, and by the categories of 
diagnosis, characteristics, and treatment. The number of test items dealing 
with each site and category may be found in Table I. 


TABLE | 


Number of Cancer Test Items Dealing with Each Site of Neoplastic Disease 
and Average Per Cent Correct for Each Class 











Number of Average Per Cent Correct for Each Class 
Site Items Freshmen Sophomores Juniors Seniors 


Skin 

Head and Neck 
Chest 

Endocrine Glands 
Digestive System 
Urinary Tract 
Male Genital 
Female Genital 
Breast 

Bone 

Soft Tissue 
Nervous System 


~ 


25 55 75 80 
31 46 63 70 
31 43 61 70 
27 52 71 80 
23 50 68 79 
16 44 70 82 
42 73 93 96 
25 40 60 70 
31 58 15 81 
19 48 58 67 
14 50 64 70 
42 58 68 73 


CONNIURPNwWNUIWU 


Lymphomas and 

Leukemia 44 58 66 68 
Unspecified 27 54 67 1 
Total—All Sites 28 52 69 76 





The average per cent correct for the sites and categories of diagnosis, 
characteristics, and treatment were computed for each year and plotted 
in the form of a learning curve. The data are presented in numerical form 
in Table II (on page 78). 
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TABLE II 


Number of Cancer Test Items Dealing with the Diagnosis, Characteristics, 
and Treatment of Neoplasms, and Average Per Cent 
Correct for Each Class 





Number of Average Per Cent Correct for Each Class 

Category Items Freshmen Sophomores Juniors Seniors 
Diagnosis 19 30 48 66 75 
Characteristics 31 23 50 67 76 
Treatment 10 37 61 76 81 
Total—All Categories 60 30 53 70 17 





Results and Analysis of Findings 


The results of the analysis are presented in Figures 1 and 2. Figure 1 
presents the curves for various sites of cancer', while Figure 2 compares 
the categories of diagnosis, characteristics, and treatment. (Normally, in 
reports to the participating schools the data are broken down further 
within each of the sites into these categories. However, in the present 
analysis, this was not feasible since the number of items would have been 
so reduced as to make the results unreliable.) 


Considering the data presented as a whole, some general statements 
of a descriptive nature may be made. All of the learning curves indicate 
that knowledge of cancer at the Freshman level, regardless of the site 
involved, tends to approximate the level of chance. On a multiple-choice 
test of this type, in which each item has five choices, a student who does 
not know the answer to an item has, theoretically, one chance in five of 
guessing the correct response; this, of course, assumes that each response 
has an equal chance of being selected. Therefore, the average per cent 
correct to be expected by chance alone is one-fifth of the total possible 
number of correct responses, or twenty per cent of the test. It is interest- 
ing to note that almost all of the curves, with several minor exceptions, 
fall near this level at the Freshman year. 


As might be expected, by one familiar with the curricula of the medical 
schools in the United States, the greatest amount of cancer learning takes 
place during the Sophomore and Junior years, particularly in courses 
dealing with pathology. The curves in the Figures seem to verify this 
trend, since the major increments in knowledge of cancer take place during 
the Sophomore and Junior years. 





1 Curves for only 7 sites are presented in this figure because of the similarity 
of thosefor all the sites. Other data not shown in this curve are found in numerical 
form in Table I 5 
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Although the majority of the curves show a definite leveling-off during 
the Senior year, this trend does not necessarily imply that less learning 
takes place during that period. One should be aware of the pitfalls which 
occur in the interpretation of the per cent statistic; namely, that it does 
not consist of equal intervals, and hence that a gain of five points from 80 
to 85 per cent represents an entirely different amount (and perhaps quality) 
of learning than a similar gain from 50 to 55 per cent. Nevertheless, the 
leveling-off of these curves is of some significance and is not entirely 
unexpected. One must remember that the greatest amount of formal 
instruction in cancer is given in the second and third years of the medical 
students’ training (primarily in pathology courses). In the Senior year 
instruction becomes much less formal and is usually concerned with clinical 
application of previously acquired principles and techniques. Therefore, 
considering the informal type of instruction involved, one would expect 
that the gains in factual knowledge would be somewhat less than at the 
lower levels of instruction in medical schools. It may also be that an 
integration of previously acquired knowledge takes place at the upper levels 
of medical education (2). 

In terms of the categories of diagnosis, characteristics, and treatment 
(Figure 2), it is of interest to note the great similarity of the learning curves 
for knowledge of diagnosis and characteristics of cancer, and the relative 
superiority shown by the curve illustrating treatment. Although one might 
consider the data for the treatment items to be somewhat unreliable when 
compared to the other two categories (because of the smaller number of 
items involved), other data on file in the Education Project tend to confirm 
this trend. One might assume that this difference may well be due to the 
fact that there tends to be less disagreement concerning the treatment of 
cancer, once a definite diagnosis has been established, than there exists 
in the actual diagnosis of the disease and its characteristics. 


Practice Effect vs. Learning 


In a study of this nature, one must be aware of the fact that the repeated 
use of test items year after year on a single group of individuals may tend 
to artificially augment the amount of gain shown. This problem is not a 
new one since any achievement testing program which uses item analyses 
to select and retain the better discriminating items for future use may 
encounter this difficulty. In a study devoted to this problem and dealing 
specifically with the Cancer Test, Towner and Galloway (3) found that 
students were able to differentiate significantly between old and new items 
in terms of recognition, but that no significant correlation existed between 
the number of old items a student correctly recognized and improvement 
in his score during three administrations of the examination, 

To investigate further the possibility of a gain due to memory for test 
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items the following analysis was made. It was hypothesized that the length 
of time during which an item had appeared on the Cancer Test would 
cause an increase in the average per cent correct of items for Senior classes 
over a four-year period. The per cent correct for all the Senior students 
taking the test in the United States during the years under consideration 
(1951-54) was determined for each of the items. Having obtained the 
average per cent correct for each item, the 1951 Senior data were then 
used as a base and gain or loss from this level was determined for each 
year. It was found that between the years 1951 and 1952 the average gain 
among these sixty items was 1.57 per cent. By the end of the second year, 
these items had, on the average, gained 3.62 per cent over 1951, and by 
1954 the average per cent gain was 4.93. On the basis of these figures one 
might expect an average gain of five per cent for an unrevised item appear- 
ing in the test over a four-year period, and this might possibly be ascribed 
to memory of items. However, it is more than likely that this variation 
represents differences in teaching, differences in student preparation for 
the test, and many other variables rather than memory of test items. As 
Towner and Galloway pointed out in their article, the students see a given 
test for less than two hours once a year and it is highly unlikely that a 
conscious effort on their part would be made to memorize the test items, 
particularly in view of the fact that none of the participating schools uses 
the Cancer Test as a means of grading students. The test, as used by 
the schools, is primarily a method of evaluating the teaching program of 
a school as a whole rather than a means of grading any individual student. 

In view of the foregoing, it would seem that the trends in the cancer 
learning of medical students as shown by Figures 1 and 2 are indicative 
of actual learning, and are not affected to any significant degree by students’ 
memory of test items. It would also seem highly probable that the average 
Senior graduating from medical schools today is better informed concern- 
ing the various aspects of cancer than was the graduate four years earlier. 


Summary and Conclusions 


This investigation is a longitudinal study of a representative, randomly 
selected sample of four hundred medical students who have taken the 
“Examination for Students of Medicine in the Subject Matter of Cancer” 
in each of the four years of their medical education. It is based on sixty 
test items which remained identical in content and wording during this 
four-year period. These items were grouped by site of neoplasm and by 
the more general categories of diagnosis, characteristics, and treatment of 
cancer. The average per cent of students responding correctly to each 
item was computed for each academic year and these results were plotted 
in the form of learning curves. The findings of this investigation may be 
summarized as follows: 





CALIFORNIA JOURNAL OF EDUCATIONAL RESEARCH Vol. VIII, No.2 
SOR a a ne a me a ace na ae RR i a tn a RN 


1. Regardless of analysis by site of cancer or by categories of diagnosis, 
characteristics, and treatment, the greatest gains in cancer learning appear 
in the Sophomore and Junior years. It was hypothesized that this was 
probably due to the instructional emphasis on pathology in these years, 


2. Knowledge of cancer during the Freshman year falls near the level 
of chance; that is, it approximates the score which might be obtained by 
a student who took the Cancer Test and guessed at all of the items. 


3. The majority of learning curves show a leveling-off at the Senior 
level of instruction. This may be due to the nature of the per cent statistic 
or it may indicate that an integration of previous cancer learning is taking 
place. 


4. Knowledge of test items dealing with the treatment of cancer exceeds 
that of items dealing with diagnosis and characteristics at each academic 
level. This may be due to the relative concurrence of opinion concerning 
the treatment of cancer, once a diagnosis has been established, as contrasted 
with less agreement regarding the diagnosis and characteristics of neoplastic 
disease. 


5. The possibility that these data might be affected by memory for 
the items was considered and investigated, and it was concluded that this 
factcr did not operate to any significant degree. 


A more detailed analysis might have been made of cancer learning by 
site of the neoplasm, but it was felt that due to the probable unreliability 
of data based on small numbers of items this step was not warranted. 
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An ABAC for Estimating Certain 
Systematic Error 


WittraM B. MIcHAEL, RosBert A. JONES, 


AND NorMAN C. PERRY 


That a systematic error exists in the estimation of levels of item difficulty 
p for a total criterion sample from an average pz of the proportions of 
individuals in the upper and lower 27 per cent-criterion groups who 
respond in a designated manner has been demonstrated both analytically 
(3) and empirically (2) when a normal correlation surface is assumed to 
describe the nature of the relationship between responses to an item and 
the criterion variable. Although the amount of error is not substantial 
for items with validities in the vicinity of .40 or less, discrepancies between 
Py and p in excess of .015 commonly occur for higher item validities when 
the level of item difficulty p, falls between .60 and .90 or between .40 
and .10. 


It is the purpose of the writers to present an abac that will permit for 
a known degree of item-criterion correlation r, an estimate of the amounts 
of error that arise in the determination of item difficulty p for a total 
criterion group when upper and lower groups, each of which consists of 
approximately 27 per cent of the total criterion sample, are employed. 
Based on use of Pearson’s Tabulation (4), the procedure for obtaining 
estimates of the amount of error has been described elsewhere (1). The 
abac is presented in Figure 1. 


If py and py, represent the proportions of individuals in each of two 
extreme groups of an entire criterion group who respond in a specified 
manner, an experimentally derived estimate of item difficulty is given by 
(py + p,)/2, which is defined as py. If p designates the actual proportion 
of individuals in the entire criteriorion sample who respond in the same 
manner, then the amount of systematic error will be given by pg — p. 
It has been shown previously that when p>50, pp — p<O and that when 
p<.50, py — p>O (3). In other words the indices pg, tend to be closer 
to .50 than do the corresponding indices p for the same degree of item- 
criterion correlation. 


A note about the authors appears on page 86. 
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FIGURE | 
An ABAC for a Graphic Estimate 


This ABAC makes possible a graphic estimate of the amounts of systematic error 
arising at different levels of item-criterion tetrachoric correlation from use of an 
index of item difficulty based on an average of the proportion of individuals in 
the upper and lower extreme groups who respond in a designated manner as an 
approximation to the index item difficulty for a total criterion sample when the 
proportion of individuals in each extreme group is approximately 0.27 (p’ = 0.2743). 
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In the abac it will be seen that along the abscissa the experimentally 
observed indices pg are shown by steps of .05. Along the ordinate the 
absolute differences |py — p| are indicated. For values in p in excess 
of .50 these differences will be negative; whereas for magnitudes in p 
below .50 the differences will be positive. Each one of the family of 
arched curves shown corresponds to a designated level of tetrachoric corre- 
lation (or equivalently biserial correlation in the instance of a normal 
correlation surface) between the responses to an item and the criterion 
variable. 

A second family of curves in p, which resembles a sequence of parallel 
lines with negative slope, permits one to determine the magnitude of p 
from knowledge of values in pg and r,. Finally it should be noted that 
the proportion p’ in each of the two extreme groups is .2743 (which is the 
proportion of area in the unit normal curve beyond a standard score of 
.60) instead of .27, since the tabulations in Pearson’s “Volumes of a Normal 
Bivariate Surface” are in terms of tenths of a standard score. 


Conclusions 


From a study of the abac it is apparent that (1) the higher the degree 
of correlation (validity) the greater is the amount of systematic error; 
(2) the amount of increase in error is positively accelerated relative to equal 
increments in r,; (3) the amount of systematic error is highly dependent 
upon the level of item difficulty as is evidenced by the fact that the magni- 
tude of the error is zero when pg is .50, is at a maximum for pg, between 
.25 and .15 (.75 and .85) when r, <.65 and for pg, between .40 and .25 
(.60 and .75) when r, >.65, and sharply drops for difficulty levels consti- 
tuting extreme departures from .50. 


lllustrative Example 


To demonstrate the use of the abac a numerical problem may be 
presented: Given that py — .29 and that r, = .52, find pg, and p and 


estimate p. Entering the horizontal axis at a point just to the right of .30, 
then rising vertically to a point about 2/5 of the way between the two 
curves corresponding to r, = .50 and r, = .55, and then following to the 
left a horizontal line (parallel to the base line), one finds on the vertical 
axis (at the left) that the amount of systematic error py — p is given as 
about |.022|. Since py <.50, the estimated amount of discrepancy pz — p 
is +.022. To determine the magnitude of p one can subtract .022 from .29 
and come out with an estimate of about .268. An alternative solution 
would be presented by looking at the point (p,,r,) = (.29, .52) and by 
then interpolating visually between the two curves (lines) p = .2743 and 
p= .2420. Since the point is roughly one-fourth to one-fifth of the distance 
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between the two parallel curves (lines), the estimate is given as .2743 + 9 
(.2420 — .2743), or about .268. 


REFERENCES 


. Hertzka, A. F., Michael, W. B., and Perry, N. C. “Systematic Error in Esti- 
mates of a Phi Coefficient,” Educational and Psychological Measurement, 12 
No. 2 (1952) 285-92). 


. Jones, R. A., and Michael, W. B. “An Empirical Study of Systematic Errors 
in Estimates of Item Difficulty Obtained from Use of Upper and Lower 27-per. 
Cent Criterion Groups.” Educational and Psychological Measurement (in 
press). 


. Michael, W. B., Hertzka, A. F., and Perry, N. C. “Errors in Estimates of Item 
Difficulty Obtained from Use of Extreme Groups on a Criterion Variable.” 
Educational and Psychological Measurement, 13, No. 4 (1953) 601-6. 


. Pearson, Karl. Tables for Statisticians and Biometricians, Part II. London 
and New York: Cambridge University Press, 1931. 


About the Authors 


William Burton Michael is Associate Professor of Psychology and Education 
and Director of the University Testing Bureau at the University of Southern Cali. 
fornia. From 1947 to 1950 he was Assistant Professor of Psychology at Princeton 
University. He was an associate professor at San Jose State College, 1950-1951, and 
a research associate with the RAND Corporation in 1951-1952. He has published 
more than fifty research papers. He received his doctorate from the University of 
Southern California in 1947. The contents of this article are based in part upona 
paper given by the senior author at the annual meeting of the California Educa 
tional Research Association, March 22-23, 1956, at Fresno. 


Norman C. Perry is Associate Professor of Mathematics at Alabama Poly. 
technic Institute. He has been with this institution since 1952 as Assistant Profes- 
sor, being promoted to his present rank in 1956. Prior to going to Alabama, he 
was an instructor at San Jose State and a lecturer at the University of Southern 
California. He obtained his doctorate from latter institution in 1952. 


Robert Anthony Jones is Lecturer in Education and Assistant Director of the 
Testing Bureau at the University of Southern California. He has held this position 
since 1950. Before coming to USC he was an instructor at George Pepperdine 
College and a teacher in the Los Angeles County Schools. 


The Selection of School Library Materials: a Guide to the Formulation of 
Policies and Procedures, sponsored by the Public Relations Advisory Pane 
of the California Teachers Association, has been prepared by the School Library 
Association of California. It contains book selection policy statements from a 
number of school districts of various sizes. The booklet is available at $.50 from 
the California Teachers Association, 693 Sutter St., San Francisco 2, California. 


Other materials on sound school library practice may be obtained from Mrs. 
Grace Dunkley of the California School Library Association, 16703 South Clark 
Avenue, Bellflower, California. 


86 





“Fade Out” of Continuation Education 


Programs in the United States 


GORDON WARNER 


The term “part time” compulsory continuation school or class had 
uniform meaning throughout the United States in 1919-1920. However, 
thirty-five years later, the conception of the term “continuation education” 
is varied. 

Compulsory continuation education consists of schools or classes for 
employed minors fourteen to eighteen years of age. While employed, these 
minors are required to attend the continuation school or class in California 
at least four hours each week. During periods of unemployment (in Cali- 
fornia) these minors are required to attend these schools or classes at least 
fifteen hours each week. 

Many boys and girls between the ages of fourteen and eighteen leave 
the regular full time school to earn the dollars they deem so necessary to 
mark them as having come of age. A review of the many studies on drop- 
outs of our regular public schools seems to indicate that dropping out is 
the result of companions’ caprices, whims and fancies, a dislike of school 
and some teachers, home life, and the desire to start working. The continu- 
ation education student is the first to admit that he is poorly prepared to 
meet the demands of society with his meager education, little knowledge 
of indusry, and lack of vocational training. 


Origin and Purpose 


Thirty-seven years ago continuation education programs afforded their 
students an opportunity to continue or receive assistance in their general 
education, obtain vocational training, and become more intelligent citizens. 

The schools aided in placement, prevented much drifting from one job 
to another, and assisted in advisement and guidance concerning vocational 
opportunities. Few juvenile misfits in the regular full time school were 


Gordon Warner is Assistant Professor of Secondary Education at Long Beach 
State College. He has served as Adviser-in-Charge, U. S. Veterans’ Administration 
Guidance Center, Los Angeles, instructor in the Laney Division of Oakland Junior 
College, and instructor-coordinator of continuation education in Oakland Unified 
School District. He received the doctorate in 1954 form the University of Cali- 
fornia at Berkeley. This article is partly based upon his doctoral dissertation. 
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assigned to the new program at first. However, they soon learned its advan. 
tages and began to find the continuation school a haven from the regular 
school routine. It was not long before the continuation education program 
became a “dumping ground” for the regular school. 

At the very beginning of part time compulsory education in the United 
States in 1919 individual states enacted laws that varied greatly. In a few 
states many conditions were stated specifically in the part time law. In 
others, the actual administration of part time schools was left to the State 
Boards of Vocational Education. In every state, either the State Board 
of Education or the State Board of Vocational Education was charged with 
the duty of formulating rules and regulations concerning the establishment 
of compulsory part time schools. Each state has made reference at some 
time to the number of minors required for establishment of a continuation 
education school or class program. 


Trends 


Continuation schools have not been established with uniformity in all 
of the states and territories, nor have the legal provisions for the establish- 
ment of continuation education programs been passed by all of the forty- 
eight states and territories. Table I indicates the wide variance and utter 
lack of uniformiy among the states with respect to attendance regulations, | 
The age group of the children required to attend has a considerable varia- 
tion in each of the states. The enforcement and the exemption provisions 
are different for a variety of reasons. The evolution of the laws governing 
continuation school education in the separate states and territories has 
been haphazard at most. In most cases the actual establishment of continu- 
ation schools preceded the laws which later made them compulsory. 

The early movements gave results which were approved by school 
administrators as can be seen from the following comments. Superin- 
tendent Stratton D. Brooks of the Boston Public Schools was authorized 
in May, 1909, “With (the) assistance of merchants and manufacturers 
in that city, to establish schools similar to the continuation schools of 
Europe,” whereby persons compelled to leave the regular day school could 
receive instruction that would enable them to become more efficient in the 
stores and factories in which they were employed. 

In New York City in 1911, a recommendation for a system of continua- 
tion schools was made in the following statement by Superintendent 
William H. Maxwell, of the New York City Public Schools: 

I recommend a system of continuation schools from 7 a.m. to 9 a.m. 
and from 4 p.m. to 6 p.m. to be organized, the legislations be sought to 
require employers to give each employee under 19 years of age, 4 to 6 
hours a week for 40 weeks each year and to constrain young people 
between these ages to attend such schools regularly. ... An effort should 


be made to adapt the work of these schools to the individual capacities 
and needs of each pupil.* 
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The World War I period gave the needed impetus to the continuation 
education movement throughout the United States. The great lack of 
industrial efficiency was apparent in so many cases that the need for an 
expanded educational program for out-of-school minors had been made 
obvious to many who had never considered the problem before. Even so, 
there was only a comparatively small group of employed minors who 
made an effort to avail themselves of the existent continuation school 
program. 

The Continuation School Teachers Association of New York City pub- 
lished an account of the daytime continuation education program of the 
city in 1927. The publication gave the total enrollment of New York City 
continuation schools as 63,600 students in 1927. There were 537 teachers 
giving instruction to part-time continuation classes. 

Although Professor Albert Shiels of Teachers College, Columbia Uni- 
versity, saw the continuation schools as an attempt to meet the needs of 
young workers, E. W. Edwards thought that criticisms of the part time 
day continuation school law and attempts to amend this law had become 
an organized movement by 1927. This movement was designed to bring 
about the substitution of evening schools for the day continuation schools. 

In 1930 over three hundred thousand students were enrolled in the 
282 remaining continuation schools in the nation.’ But a review of the 
Digests of Annual Reports of State Boards for Vocational Education for 
the years following this shows that the continuation education school enroll- 
ment was trending downward from its peak in 1930. 

The opinion of continuation education leaders as of 1949 is found in 
the statements received by Smith-Jacobs from prominent authorities on 
the future of the continuation program: 


Keller: The decline in interest in continuation education has cer- 
tainly been due to the raising of the full time compulsory school 
age... 


Evans: Continuation school education has served its purpose for the 
14-16 year age group because practically all of that group is now 
required to attend school until sixteen. 


Rasche: First, practically all our large cities are embarrassed by 
diminishing tax resources. . . . These schools (continuation) should 
be abandoned only when the general school system actually achieves 
in doing everything that should be done for youth through education 
as influenced by organized schools. 


—__ 


‘Grayson N. Kefauver, Victor H. Noall, and C. E. Drake, Par’-Time Secondary 
Schools, Monograph No. 3, Bulletin No. 17, Washington, D.C., U.S. Office of Eda- 
cation, Government Printing Office, 1932. 
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The Question 


To obtain information about the status of continuation education in the 
forty-eight states and seven territories today, a brief questionnaire was 
formulated in March 1954 and sent to each State Superintendent of Edu- 
cation. All questionnaires sent out were returned, so that a complete 
survey of the status was possible. See Table I for the tally of the questions, 
The data were compared with the results of earlier surveys to determine 
trends. 

Answers to the questionnaire regarding the operation of a continuation- 
education or part-time-education program for working minors disclosed 
that some states have ceased to offer a continuation program, while others 
have not as yet established either a class or school for their working minors. 
From the loose interpretation of the term “continuation education” or part 
time continuation, it can be concluded from the results of the questionnaire 
that there is a decline in the interest for such a program today. 


California's Problem 


The enthusiasm which swept California for part time education lasted 
just ten years from the enactment of the law in 1919. In 1930, legislation 
revised the part time law both in purpose and title. The program was now 
compulsory, and changed to continuation education. Thus, any student 
who had not completed regular full time high school requirements for 
graduation and who received a permit to work was required to attend a 
continuation school or class. 

In addition to regularly employed students who obtained a work permit, 
students who were unemployed also were required to attend a continuation 
school or class not less than 3 hours per day, each day, until they obtained 
work, This section of the new law brought a wave of resentment from 
parents and youth, 

From many studies which have been made it is apparent that the part 
time school population is a selected group that is quite diversified, and 
generally having intelligence quotients that overlap with pupils who remain 
in the regular school. However, generally speaking, the average mental 
age of the continuation school student has been found to be lower than 
the average of the equivalent group in the full time school. This wide 
range requires individual material to be constructed to fit the various levels. 
Among these students the majority have fallen far below the grade for their 
age, and the habit of failure is overwhelmingly strong. It is exceedingly 
difficult to rebuild confidence. 

The instructor in a continuation education class is constantly faced 
with the problem of avoiding repetition. Students in continuation education 
face blocks which regular high school students do not meet. Compulsory 
attendance, work permits for each new job, termination cards, regulations, 
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TABLE | 


Status of Continuation Education Programs in the United States 
and Its Territories in 1954* 


Place and Date Where Number of Number of Is Program New Program 
First Established Students, Teachers Adequate? Needed? 
1954 1954 


ALABAMA Good program, 

Birmingham, 1918-20 1,945 41 poor finance@ 
CALIFORNIA 

Los Angeles, 1918 4,962 seal Yes & No 
FLORIDA—Miami, 1948 8,345 383 Yes 
CGEORGIA—Atlanta, 

Porterdale, 1919 2,685 48 Yes 
ILLINOIS—Chicago, 1919 ses ees sam 
INDIANA—Wabash, 1917 563 Yes & No 
IOWA—Mason City, 1918 2,799 oi Yes & No 
MARYLAND 

Baltimore, 1924, 1936 3: No 
MASSACHUSETTS 

, 1913 ae : 
MICHIGAN—Detroit, 1919 No 
MINNESOTA— 5 ih Yes 
MISSOURI 

Columbus City, 1936 7 Yes No 
NEW YORK 

Manhattan, Bronx, 1910 : eau Noe Noe 
OKLAHOMA—Tulsa, 1935 : 34 Yes Small areas 
OREGON—Portland, 1925 § os Yes 
PENNSYLVANIA 

Philadelphia, 1925 Af y Yesd No 
SOUTH CAROLINA 

Youngs Township, 1917 : d ; Yes 
TEXAS—Laredo, 1854 vs os, 
VIRGINIA 

Norfolk Navy Yard, 1923 ; : les e 
WASHINGTON 

Seatile, 1922 20 Se ae No 
WISCONSIN—Racine, 1911 11,348 527 ae No 
WYOMING—Casper, 1948 65 Yes ass 
GUAM— » 1954 ae - 














4Set up for workers in trade and industry. Program operates only in Chicago. 
Reorganization of curriculum needed. Needs expansion as there are 40,000 still 
not attending. More divisions are needed. 


. Continuation Education programs are not in progress at the present time in the 
following states and territories: 


Arizona Kansas Nebraska North Dakota West Virginia 
Arkansas Kentucky Nevada Ohio Alaska 
Colorado Louisiana New Hampshire Rhode Island Hawaii 
Connecticut Maine New Jersey South Dakota Panama 
Delaware Mississippi New Mexico Utah Puerto Rico 
Idaho Montana North Carolina Vermont Samoa 

Virgin Islands 
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Results 


It is felt by some administrators that there is a decreasing need for 
continuation education schools and classes with the decreasing employment 
opportunities for minors. The increased holding power of the regular 
full time school has also been a factor in lowering the enrollment of 
continuation schools. 

Generally only a few employers will give the under-eighteen-year-old 
youth an oportunity to work because of the city, county, state, and federal 
regulations which must be followed during the youth’s employment. A 
major deterring factor is the cost of the insurance which must be held by 
the employer. 

There is a change in continuation schools in many states in the directior 
of becoming adjustment schools to handle the problem cases of the regular 
full time school. It is the intent of these adjustment schools to attempt to 
direct the student who has dropped from a regular school program back 
into the regular full time school program with a much better understanding 
of the requirements of society or drop the student from the school program. 

There is need for increased state financial support for continuation 
education schools or classes. If the revenue is not forthcoming, it would 
be advantageous not only to the student, but the school districts as well, 
to consolidate continuation education with adjustment-schools and support 
the enlarged program. 

There are wide variations among attempts to coordinate the special 
services offered by continuation education with other secondary school 
services. More knowledge of the community and its resources is required 
of the continuation education coordinator, who is most always the instructor 
of the continuation class, than is required in the elementary or secondary 
schools. 

The method and content of instruction vary widely. There are some 
cities which conduct unit programs; others require regular standardized 
high school work; and others have a greatly modified general continuation 
education program. In some cities the continuation education classes are 
furnished with a permanent projector for visual aids, and an adequate 
library stocked with current magazines and other material. This is highly 
recommended because of the variation of continuation student interest. 

Work permits today are merely an additional holding device affecting 
a very small percentage of young drop-outs from public, parochial, and 
private schools. Through various interpretations of the legal sections on 
compulsory continuation education throughout the nation, the continuation 
student is largely directed at the local level. A small number of the sixteen- 
to - eighteen - year-olds eligible for continuation education are actually 
enrolled in such classes. Those that are enrolled believe that they are 
caught in an unjust situation. 
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There is a very definite need for the simplification and clarification of 
the California compulsory continuation education law. At the present time 
it is being interpreted in different ways by different legal authorities. If there 
is to be a continuation of the California compulsory education program con- 
structive action should be taken to correct its major defects. If there is to be 
a law to govern continuation education in the state, the law should be 
adhered to in every county and city of California, and not left up to local 
interpretation. The law should be so drawn up that its intent can be under- 
stood by administrators, instructors, the students, parents, and employers. 

It is recommended that the compulsory continuation education law be 
amended in the matter of continuation school attendance so that such 
attendance as is now required be eliminated and in its place be substituted 
a program of guidance and follow-up of sixteen-year-olds who may be 
employed. The continuation schools were organized under conditions 
which no longer prevail, both in the working world and in education. It 
is questionable whether the four hours a week of compulsory attendance 
is achieving its purpose despite the well intentioned efforts of those who 
are responsible for its administration. 

The excellent cooperative relationship between schools, parents and 
employers was generally overlooked by the student. Through the work per- 
mit form, all three had a hand in the youth’s future. His place of employ- 
ment was known; the hours of school attendance were recorded; and the 
parent gave approval. This combination afforded him the opportunity of 
obtaining experience and training for his future. This arrangement lasted 
until twenty-five years ago when, as a result of the depression, youth was 
forced to leave employment. The continuation education school lost its 
enrollment and gradually was absorbed by the trade and industrial schools. 

The continuation education program where it exists is a social labora- 
tory in which the vocational, mental, physical, moral, and social equipment 
of the young worker are analyzed. This is carried out with a view to the 
development of special abilities and the correction of deficiencies as they 
are revealed. Primarily, the continuation school means simply the continu- 
ing of the young worker’s education as far as possible. 

If a continuation program is to exist the community must join with the 
public school in establishing sound employment programs for workers 
under the age of eighteen. Interested organizations should pool their 
possible job openings and afford young workers the opportunity of partici- 
pating in gainful employment. 

This project should have attached to it a sound vocational guidance 
program. Such a program could effectively coordinate the employment of 
sixteen-to-eighteen-year-old youth. Such a program would not only guide 
youth into feasible vocational fields for future development, but could 
detect the maladjusted, and protect and assist both the employee and 
employer in a trying situation. 
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Book Reviews 


A STUDY OF THE NEED FOR ADDITIONAL 
CENTERS OF PUBLIC HIGHER EDUCATION IN CALIFORNIA 


Sacramento: California State Department of Education, 1957. 172 pages, paper. 


This publication constitutes the report of the staff of the Liaison Com- 
mittee of the State Board of Education and the Regents of the University 
of California. This Committee was esablished in 1945 as a means of 
coordinating the University and the state colleges. It consists of four 
members each from the Regents and the State Board of Education, includ- 
ing the President of the University and the Superintendent of Public 
Instruction. 


The 1955 Legislature received many bills calling for the establishment 
of new state colleges. As a result, legislation was enacted directing that 
studies be made of the needs for higher educational institutions in the 
State, with particular reference to areas in which new state colleges have 
been proposed. The Liaison Committee employed a joint technical staf 
to survey the needs and prepare a report. 


The staff and the Committee are to be congratulated upon the broad 
scope of their report. They have not confined their study to the areas where 
study was required by the legislature, nor have they restricted it to the 
University and state colleges. The need for junior colleges is given major 
consideration, along with the probable interactions of colleges, university, 
and junior colleges. 


Major attention is given to the recommendation of criteria for the 
establishment of junior colleges, state colleges, and new campuses of the 
University of California. The report indicates that more of all three facili- 
ties are needed. 


Fourteen new junior college districts are recommended. The report 
strongly implies that these should be created at once, and notes approvingly 
that elections have been called in connection with two of them. Two of 
the proposed districts would relieve the remaining state colleges that have 
junior-college functions from this responsibility. It is the opinion of the 
staff and the Committee that it is not possible satisfactorily to combine the 
functions of a state college with those of a junior college. 


Nine new state colleges are recommended for serious consideration by 
1970. State colleges in Alameda, San Bernardino or Riverside, Contra 
Costa, and Kern Counties ‘are recommended for immediate consideration. 
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Five others are indicated as having enrollment potentialities sufficient to 
warrant their establishment by 1970. Four other possible locations for 
state colleges are given negative recommendations on the grounds that 
their enrollment potential is not great enough in the foreseeable future. 
The recommendations are based on the assumption that an enrollment of 
at least 2,000 full-time students is needed to make unit costs reasonable. 
Dormitory and transportation facilities, together with state scholarship 
aid, are recommended to take care of students from areas in which state 
colleges cannot be economically established. 


Since the projected enrollments of Berkeley and UCLA are 35,200 and 
39,900, respectively, by 1970, the report strongly recommends additional 
campuses for the University of California. Southeast Los Angeles County, 
Santa Clara County, and San Diego County are the three areas in which 
immediate expansion of facilities is recommended. The San Diego campus 
might be formed by an expansion of the present Scripps Institute of 
Oceanography into a full-fledged general university campus; but the other 
two locations would require entirely new establishments. 

In addition to specific recommendations, the report contains much 
interesting material on the functions of the three types of instituions consid- 
ered, the nature and quality of the programs they are now offering, and 
the problems they are facing in adjusting to expanding enrollments. The 
book concludes with a very comprehensive appendix that contains all the 
statistics and data on which the recommendations were based. Whether 
or not one agrees with the opinions expressed, one must recognize the 
report as being very sound from a research point of view. 


CANADIAN EDUCATION TODAY, A Symposium 
JOSEPH KATZ, Editor 





Toronto: McGraw-Hill Book Company of Canada Limited, 1956. 243 pages. $3.95. 


Canadian education is a lot like education in the United States; but it 
is by no means the same. This book is not an attempt to explain either the 
similarities or the differences to those south of the border, but it will be 
interesting and very instructive to those United States educators who read 
it. They will find the usual chapters on education and social change, 
educational psychology, curriculum construction, teacher training, and so 
on; but they will also find chapters on the “new composite schools,” the 
work of the universities, and the humanities, which approach things in a 
different way. 

The greatest similarity between Canadian and United States education 
is the absence of a centralized federal system of schools. As the editor says 
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in the summary chapter, “Canada is possessed of educational systems which 
are vigorous in their pursuit of different ideals of education, are vital in 
their approach to the many different problems confronting schools and 
universities; and are as different from one another as are the provincial 
boundaries which separate them.” 

Differences, on the other hand, are to be found more in the degree of 
emphasis given to various goals of education and methods of procedure, 
The role of the school in the development of children is plainly thought to 
be less vital than is the case in the United States. In particular, there is a 
greater tendency to view education as preparation for life and to expect 
early differentiation based on the life goals of the student. However, the 
composite high school movement described in one chapter may reduce 
this tendency in the future. 


EDUCATION AS A PROFESSION 


MYRON LIEBERMAN 
Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1956. 590 pages. $8.00 

This book is the first comprehensive survey of the professional status 
of education. The author appears to have done an excellent job of report- 
ing on the present conditions in the occupation of education. It is his 
contention that the assumption that education is already a profession has 
prevented an objective analysis of the degree to which it actually meets 
the criteria of professionalism. 

The first part of the volume is devoted to a description of the character- 
istics of a profession. Beginning with the fifth chapter various educational 
activities are studied in terms of this description. Areas considered include: 
teacher education and certification, educational associations and _ teachers 
unions, collective bargaining, ethics, and occupational status of teachers. 

On the whole the author appears to have done an excellent job. His 
objective approach is refreshingly in contrast to the writings of some who 
smugly assume that the professional status of education is accepted and 
its pattern fixed. His chief weakness appears to result from his lack of 
awareness of some of the most recent advances in professionalism in certain 
areas of education, such as professional standards, salary and personnel 
procedures, development of educational policy. His treatment of state and 
local teacher organizations—both union and professional—is also weak. 

This book is recommended to all who seriously aspire to creating a 
profession of education and to all others who recognize that the professional 
status of education profoundly affects their own position. It is to be hoped 
that its shortcomings will inspire others to follow the author’s lead by 
producing some other books which will emphasize the advances made 
recently in California and other states. 
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